Marker processing method, marker processing device, marker, object having a marker, and marker processing program

ABSTRACT

A marker processing method includes: (a) binarizing a shot image; (b) labeling one or more constituents of the image detected based on the image binarized in step (a); (c) obtaining a region centroid of each of the constituents corresponding to the respective labels processed in step (b); (d) obtaining a degree of overlap of the region centroids of the constituents corresponding respectively to the labels, obtained in step (c); and (e) detecting a marker based on the degree of overlap of the region centroids obtained in step (d).

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a divisional patent application of U.S. application Ser. No.12/833,250 filed Jul. 9, 2010 which claims priority to Japanese PatentApplication Nos. 2009-179265 filed Jul. 31, 2009, 2010-064322 filed Mar.19, 2010 and 2010-149715 filed Jun. 30, 2010 all of which are expresslyincorporated by reference herein in their entireties.

BACKGROUND

1. Technical Field

The present invention relates to a marker processing method, a markerprocessing device, a marker, an object having a marker, and a markerprocessing program.

2. Related Art

As a method of detecting a marker out of an image obtained by shootingan object attached with the marker there are known a method of detectinga symmetrical property of the shape and a method of detecting acombination of colors.

As a method of detecting a symmetrical property of a shape, there isproposed a method of detecting a two-dimensional code having apositioning symbol. The positioning symbol is disposed at apredetermined position, and the location and the rotational angle of thetwo-dimensional code can be obtained using the positioning symboldetected in the image thus shot (see, e.g., JP-A-7-254037 (Document 1)).

As a method of detecting a combination of colors, there is proposed amethod of detecting a hue region entirely surrounded by a different hueregion as a marker. The two hue regions used as a marker are previouslyprovided with an identification number for each combination of colors.Therefore, a hue image is extracted from the shot image, and then avariation pattern in the hue is searched by scanning from the hue imagethus extracted. By detecting the region, which can be expected to be themarker, using the hue search described above, and then determiningwhether or not the variation pattern of the hue thus detected matchesthe predetermined combination, the marker is detected (see, e.g.,JP-A-2005-309717 (Document 2)).

However, according to the method of detecting the symmetrical propertyof the shape using the technology described above, since the positioningsymbol, in which a ratio of dark and bright periods is set asdark:bright:dark:bright:dark=1:1:3:1:1 as shown in FIG. 2, is detectedby scanning, there arises a problem that the detectable range of thesymbol, which is rotated or tilted, becomes narrower depending on thescanning interval. Further, since a high symmetrical property isrequired for the marker itself in order to cope with the cases in whichthe scanning line for detecting the marker traverses the marker invarious directions, which problematically causes restriction on creatinga number of markers. Further, since it is only required that the darkand bright periods have the ratio of 1:1:3:1:1, and there is basicallyno limitation on the absolute periods, there arises a problem that themarker detection side is required to cope with the period variation dueto the size of the marker. Further, since the black/white inversionperiod is used as the marker, determination of the period becomesdifficult when noise is mixed into the input image. Therefore, therearises a problem that some measure against the noise becomes necessary.

In other words, the technology of the Document 1 has a problem that themarker detection depends on the posture (position, rotation, or tilt) ofthe marker, depends on the size of the marker, and is further influencedsignificantly by the noise in the image.

Further, in the method of detecting a color combination according to thetechnology described above, it is required to perform the dataprocessing with an amount roughly three times as large as that in thecase of using a monochrome image. Therefore, there arises a problem thatit is required to reduce the resolution of the image or to reduce theframe rate when capturing the image in order for achieving the amount ofprocessing equivalent to that in the monochrome image. Further, sincethe hue information in the shot image is significantly influenced byillumination conditions and so on, and is further influencedsignificantly by the white balance and so on of the camera used forshooting, there arises a problem that some countermeasures against thesefactors become necessary. Further, since the pigment or the colormaterial in the material constituting the marker to be used variesacross the ages, there arises a problem that some countermeasuresagainst the aging become necessary.

SUMMARY

An advantage of some aspects of the invention is to provide a markerprocessing method, a marker processing device, and a marker eachindependent of the posture (position, rotation, or tilt) of the marker,independent of the size of the marker, resistant to the noise in theimage, and capable of reducing the amount of processing for detectingthe marker using the monochrome image instead of the hue information.

A marker processing method according to an aspect of the inventionincludes the steps of (a) binarizing a shot image, (b) labeling one ormore constituents of the image detected based on the image binarized instep (a), (c) obtaining a region centroid of each of the constituentscorresponding to the respective labels processed in step (b), (d)obtaining a degree of overlap of the region centroids of theconstituents corresponding respectively to the labels, obtained in step(c), and (e) detecting a marker based on the degree of overlap of theregion centroids obtained in step (d).

It should be noted that the constituent of the image denotes a point, aline, or a figure included in the shot image and having the area, theregion centroid denotes the centroid (the center of figure can also beadopted) of the labeled figure, and the degree of overlap of the regioncentroids denotes the number of labeled regions having the centroids(the center of figure can also be adopted) of falling within apredetermined range.

Further, according to another aspect of the invention, in the markerprocessing method of the aspect of the invention described above, thereis further provided the step of (f) identifying a type of the markerdetected in step (e) using at least one of the degree of overlap of theregion centroids obtained in step (d), an area ratio between the regionsof the marker, and a ratio of a size between the regions of the marker.

It should be noted that the marker determination process corresponds torecognizing which is the marker in the shot image, and the markeridentification process corresponds to identifying the type of the markerin the shot image.

Further, according to still another aspect of the invention, in themarker processing method of the aspect of the invention described above,in step (e), the marker is detected if the degree of overlap of theregion centroids is one of equal to and larger than 3.

Further, according to yet another aspect of the invention, in the markerprocessing method of the aspect of the invention described above, themarker includes at least three figures having a common centroid.

Further, according to still yet another aspect of the invention, thereis provided a marker processing device including a binarization sectionadapted to binarize a shot image, a labeling section adapted to detectone or more constituents of the image based on the image binarized bythe binarization section, and label the constituents detected, a regioncentroid obtaining section adapted to obtain a region centroid of eachof the constituents corresponding to the respective labels processed bythe labeling section, a region centroid multiplicity obtaining sectionadapted to obtain a degree of overlap of the region centroids of theconstituents corresponding respectively to the labels, obtained in theregion centroid obtaining section, and a marker determination sectionadapted to detect a marker based on the degree of overlap of the regioncentroids obtained in the region centroid multiplicity obtainingsection.

Further, according to further another aspect of the invention, there isprovided a marker including at least three figures having a commoncentroid.

Here, having a common centroid denotes that the centroids of the figuresfall within a predetermined range.

Further, according to a further aspect of the invention, in the markerof the aspect of the invention described above, additional informationis further provided.

It should be noted that the marker provided with additional informationdenotes the marker embedded with redundant data generated by the typicaltwo-dimensional code generation method by superimposing the redundantdata on the marker.

Further, according to a still further aspect of the invention, in themarker of the aspect of the invention described above, the additionalinformation is digital data.

Further, according to a yet further aspect of the invention, there isprovided an article of manufacture having the marker of the aspect ofthe invention described above.

According to a furthermore aspect of the invention, there is provided amarker processing program adapted to allow a computer to execute aprocess according to an aspect of the invention, the process includingthe steps of (a) binarizing a shot image, (b) labeling one or moreconstituents of the image detected based on the image binarized in step(a), (c) obtaining a region centroid of each of the constituentscorresponding to the respective labels processed in step (b), (d)obtaining a degree of overlap of the region centroids of theconstituents corresponding respectively to the labels, obtained in step(c), and (e) detecting a marker based on the degree of overlap of theregion centroids obtained in step (d).

According to the aspects of the invention, since it is arranged that thecentroid (the center of figure can also be adopted) of each of theregions labeled from the shot image is obtained, and the marker isdetected based on the degree of overlap of the centroids of the regionscorresponding respectively to the labels, it becomes possible to providea marker processing method, a marker processing device, a marker, anobject having the marker, and a marker processing program each of whichis independent of the posture and the size of the marker, highlyresistant to the noise in the image, and allowing reduction of an amountof processing for marker detection by using a monochrome image insteadof hue information.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIGS. 1A through 1D are diagrams showing some examples of the markeraccording to a first embodiment of the invention.

FIG. 2 is a diagram for explaining constituents of the marker shown inFIG. 1B according to the first embodiment.

FIG. 3 is a diagram for explaining centroids of the constituents of themarker shown in FIG. 1B according to the first embodiment.

FIG. 4 is a block diagram showing an example of a configuration of amarker processing device according to the first embodiment.

FIG. 5 is a diagram showing an example of an image including detectionobjects attached with the markers according to the first embodiment.

FIG. 6 is a flowchart of a processing method according to the firstembodiment.

FIG. 7 is a flowchart of preprocessing according to the firstembodiment.

FIG. 8 is a diagram showing an example of the data showing coordinatesof the centroids of the respective labels obtained by a region centroidobtaining section according to the first embodiment.

FIG. 9 is a diagram showing an example of a marker output according tothe first embodiment.

FIG. 10 is a diagram showing an example of a data configuration of amarker candidate list stored in a marker candidate region list storagesection 109 according to the first embodiment.

FIG. 11 is a diagram showing an example of image information obtained bybinarizing an input image according to the first embodiment.

FIG. 12 is a diagram for explaining a labeling process according to thefirst embodiment.

FIG. 13 is a diagram showing an example of a result of obtaining thecentroids of the regions having the same label according to the firstembodiment.

FIGS. 14A through 14C are diagrams for explaining the fact that themarker detection according to the first embodiment does not depend onthe posture.

FIGS. 15A through 15C are diagrams for explaining the fact that themarker detection according to the first embodiment does not depend onthe size.

FIGS. 16A through 16F are diagrams showing examples of other markersaccording to the first embodiment.

FIGS. 17A through 17F are diagrams showing examples of a markerincluding redundant portions according to the first embodiment.

FIGS. 18A through 18D are diagrams showing examples of a handwrittenmarker according to the first embodiment.

FIG. 19 is a block diagram showing an example of a configuration of amarker processing device according to a second embodiment of theinvention.

FIG. 20 is a flowchart of a processing method according to the secondembodiment.

FIG. 21 is a diagram showing an example of centroid data obtained in thepreprocessing according to the second embodiment.

FIGS. 22A through 22L are diagrams for explaining an example ofidentifying the marker based on the difference in multiplicity betweenthe markers according to the second embodiment.

FIGS. 23A through 23L are diagrams for explaining an example ofidentifying the marker type using the area ratio, the ratio of the sizeof the region of the marker according to the second embodiment.

FIG. 24 is a flowchart of a process of embedding additional informationin the marker according to a third embodiment of the invention.

FIG. 25 is a diagram for explaining a method of creating a markerattached with a protective region according to the third embodiment.

FIG. 26 is a diagram for explaining a method of attaching data to themarker according to the third embodiment.

FIGS. 27A through 27E are diagrams showing examples of a region of themarker according to the third embodiment, where the additional data canbe embedded, and examples of actually embedding the additional data.

FIGS. 28A and 28B are diagrams showing some examples of an object havingthe marker according to a fourth embodiment of the invention.

FIG. 29 is a diagram showing an example of an object having the markeraccording to a fifth embodiment of the invention.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, some embodiments of the invention will be explained withreference to FIGS. 1A through 29. It should be noted that the inventionis not limited to the embodiments described below, but can variously bemodified within the scope or the spirit of the invention.

First Embodiment

FIGS. 1A through 1D are diagrams showing some examples of a markeraccording to a first embodiment. In FIGS. 1A through 1D, each of themarkers shown in FIGS. 1A through 1D is formed so that the centroidpositions of the respective labels in each of the markers are identicalto each other. Further, the markers shown in FIGS. 1A through 1D areeach provided with four centroids.

FIG. 2 is a diagram for explaining constituents of the marker shown inFIG. 1B according to the first embodiment. The marker shown in FIG. 1Bis constituted by a constituent 1, a constituent 2, a constituent 3(black circle), and a constituent 4 (white circle) as constituentsseparated in accordance with white and black colors. These constituentsare labeled in a labeling process described later for each regioncorresponding to the constituent.

FIG. 3 is a diagram for explaining centroids of the constituents of themarker shown in FIG. 1B according to the first embodiment. The markershown in FIG. 1B is formed having the centroids of the FIGS. 1 through 4as the constituents overlapping with each other so as to be identical ata position indicated by the symbol a as shown in FIG. 3. It should benoted that although the centroid is indicated using a cross (x) mark forthe sake of explanation, the constituents of the marker do not have thecross (x) mark in FIG. 3. In other words, the FIGS. 1 through 4 as theconstituents are arranged so that the centroids thereof are common. Itshould be noted that the relationship between the coordinates of thecentroids of the FIGS. 1 through 4 as the constituents is defined asidentical even in the case in which an error within a predeterminedrange is provided due to the resolution of a camera used for shootingthe marker, the size of the marker, or the resolution of a printer usedfor printing the marker. It should be noted that the centroid can be thecenter of figure.

FIG. 4 is a block diagram showing an example of a configuration of themarker processing device according to the first embodiment. The markerprocessing device 100 is composed of an image data acquisition section101, a binarization section 102, a binarization threshold settingsection 103, a region labeling section 104, a region centroid obtainingsection 105, a region centroid multiplicity obtaining section 106, amarker determination section 107, a marker candidate region list storagesection 108, and a marker position output section 109. Further, themarker processing device 100 receives the image shot by the camera 120.Further, the marker processing device 100 outputs the marker informationdetected by the processing to the image display device 121.

The camera 120 is composed, for example, of a light-receiving lens and aCCD camera, and shoots the image including the detection object attachedwith the marker, and then transmits the shot image to the markerprocessing device 100.

The image data acquisition section 101 acquires the image, which is shotby the camera 120, at a predetermined timing, and then outputs it to thebinarization section 102 and the binarization threshold setting section103. Regarding the acquisition timing of the image, it is possible toperform the acquisition every time the marker determination performed,or to acquire the image every predetermined period.

The shot image is input to the binarization section 102 from the imagedata acquisition section 101. Further, the binarization section 102binarizes the image thus received using a threshold value set by thebinarization threshold setting section 103, and then outputs the imageinformation thus binarized to the region labeling section 104.

The shot image is input to the binarization threshold setting section103 from the image data acquisition section 101. Further, thebinarization threshold setting section 103 sets the threshold value usedwhen performing the binarization on the image thus received, and thenoutputs the threshold value thus set to the binarization section 102.

It should be noted that as the method of setting the threshold, forexample, the method proposed by Nobuyuki Otsu in 1979 (hereinafterreferred to as Otsu's method) is used.

The binarized image information is input to the region labeling section104 from the binarization section 102. Further, the region labelingsection 104 performs the labeling process of the regions by a typicallabeling method on the binarized image information thus received, andthen outputs labeling information of the region to the region centroidobtaining section 105. According to the labeling process, the binarizedimage information is separated into figures constituting the image.

It should be noted that as the method of the labeling process, there canbe cited a 4-neighbor process, an 8-neighbor process, and so on, whichhave already been known to the public, and therefore, the explanationtherefor will be omitted.

Further, the labeling information of the region corresponds to theinformation obtained during the labeling process, such as a total numberof pixels constituting the region thus labeled, the maximum value andthe minimum value of the X coordinate and the maximum value and theminimum value of the Y coordinate in the region, and a label numberprovided to the region separated by the labeling process.

The labeling information of the region is input to the region centroidobtaining section 105 from the region labeling section 104. Further, theregion centroid obtaining section 105 obtains the coordinate of thecentroid for every region by a typical method using the labelinginformation of the region thus received, and then outputs theinformation of the centroid coordinate obtained to the region centroidmultiplicity obtaining section 106.

The information of the centroid coordinate of each of the regionsobtained from the region centroid obtaining section 105 is input to theregion centroid multiplicity obtaining section 106. Further, the regioncentroid multiplicity obtaining section 106 compares the positions ofthe centroid coordinates of the respective regions using the informationof the centroid coordinates of the respective regions thus received. Asa result of the comparison, if the centroid coordinates thereof fallwithin a predetermined tolerance, the region centroid multiplicityobtaining section 106 determines that the centroids are located at thesame coordinate, and obtains the multiplicity as a degree of overlap ofthe centroid. The multiplicity as the degree of overlap of the centroiddenotes the number of centroids of a plurality of figures falling withina predetermined tolerance. For example, in the case in which a centroidA of a figure A and a centroid B of a figure B fall within apredetermined tolerance, the multiplicity is obtained as 2. Further, thetolerance for determining whether or not the centroid coordinatesoverlap with each other is set based on, for example, the focal lengthand the resolution of the camera 120 used for shooting, the centroidposition accuracy in forming the marker, and the resolution of a printerfor printing the marker.

The multiplicity of the centroid coordinates of the respective regionsobtained from the region centroid multiplicity obtaining section 106 isinput to the marker determination section 107. Further, the markerdetermination section 107 reads out information of a marker candidateregion described later stored in the marker candidate region liststorage section 108. Further, the marker information corresponds to, forexample, the label number, the centroid coordinate, and themultiplicity. Further, the marker determination section 107 determineswhether or not the centroid coordinate has the multiplicity equal to orgreater than a predetermined multiplicity using the multiplicity of thecentroid coordinates of the regions received from the region centroidmultiplicity obtaining section 106 and the information of the markercandidate region read out from the marker candidate region list storagesection 108. Further, in the case in which the multiplicity is equal toor higher than a predetermined value as a result of the determination,the marker determination section 107 determines it as the marker, andoutputs the marker information of the region determined as the marker tothe marker position output section 109. The multiplicity in the markerdetermination is, for example, 3 or higher.

The marker candidate region list storage section 108 stores the centroidcoordinates of the region 1 and region 2, and the multiplicity thereofstored by the region centroid multiplicity obtaining section 106.

FIG. 10 is a diagram showing an example of a data configuration of amarker candidate list stored in a marker candidate region list storagesection 108 according to the first embodiment. As shown in FIG. 10, themarker candidate region list is stored having the label numbers, thecentroid coordinates, and the multiplicity of the centroid correlatedwith each other.

The marker information thus determined is input to the marker positionoutput section 109 from the marker determination section 107, and themarker position output section 109 generates the information displayedon the image display device based on the marker information thusreceived, and then outputs it to the image display device 121.

FIG. 9 is a diagram showing an example of the marker information outputby the marker position output section 109. As shown in FIG. 9, themarker information output by the marker position output section 109includes the centroid coordinate of a figure determined as the marker,the multiplicity of the centroid, the label name as the information ofthe regions having the centroids overlapping with each other, and so oncorrelated with each other.

The image for displaying the marker information thus generated is inputto the image display device 121 from the marker processing device 100,and the image display device 121 displays the image thus received.

FIG. 5 is a diagram showing an example of an image including detectionobjects attached with the markers according to the first embodiment. Inthe example shown in FIG. 5, there are four detection objects, each ofwhich is attached with either one of the markers shown in FIGS. 1Athrough 1D. Further, as shown in FIG. 5, the markers of the respectivedetection objects attached with the markers are different in position,and are disposed with rotation, magnification or reduction due to theinfluence of the perspective caused by the arrangement.

Then, the marker processing method according to the first embodimentwill be explained using the flowcharts shown in FIGS. 6 and 7, and anexample of the centroid data shown in FIG. 8 obtained by apreprocessing. FIG. 6 is a flowchart of the marker processing methodaccording to the first embodiment. FIG. 7 is a flowchart of thepreprocessing in the first embodiment. FIG. 8 is a diagram showing anexample of the data showing coordinates of the centroids of therespective labels obtained by the region centroid obtaining section 105.Firstly, the centroid of each of the regions is obtained (step S1) inthe preprocessing.

The preprocessing in the step S1 will be explained using the flowchartshown in FIG. 7. The image acquisition section 101 obtains (an imageacquisition process: step S101) the image shot by the camera 120.

The image acquisition section 101 outputs the image thus acquired to thebinarization section 102 and the binarization threshold setting section103. The binarization threshold setting section 103 obtains (abinarization threshold setting step: step S102) the threshold value forperforming the binarization based on the image received from the imageacquisition section 101 using, for example, the Otsu's method.

Subsequently, the binarization section 102 binarizes (an imagebinarization process: step S103) the image received from the imageacquisition section 101 using the threshold value set by thebinarization threshold setting section 103. The binarization section 102outputs the image information thus binarized to the region labelingsection 104.

Subsequently, the region labeling section 104 performs (a regionlabeling process: step S104) labeling of the region based on thebinarized image information received from the binarization section 102.Further, the region labeling section 104 outputs the information thuslabeled to the region centroid obtaining section 105.

Subsequently, the region centroid obtaining section 105 obtains (aregion centroid obtaining process: step S105) the coordinate of thecentroid of each of the label regions from the labeled informationreceived from the region labeling section 104. The region centroidobtaining section 105 outputs the coordinate of the centroid of each ofthe label regions thus obtained to the region centroid multiplicityobtaining section 106. Then the preprocessing is terminated.

Going back to FIG. 6, the region centroid multiplicity obtaining section106 and the marker determination section 107 determine (step S2) whetheror not the processing of all of the unprocessed regions has beencompleted. The completion of the processing of all of the unprocessedregions is determined based on whether or not the obtaining of themultiplicity of all of the combinations of the labeled regions by theregion centroid multiplicity obtaining section 106, and the markerdetermination of all of the combinations of the regions by the markerdetermination section 107 have been completed.

If it is determined in the step S2 that the processing of all of theunprocessed regions has not been completed (No in the step S2), theregion centroid multiplicity obtaining section 106 deletes (step S3) thedata of the list of the marker candidate regions stored in the markercandidate region list storage section 108 to empty the marker candidateregion list storage section 108.

Subsequently, the region centroid multiplicity obtaining section 106selects (step S4) one unprocessed region 1 out of the coordinates of thecentroids of the respective label regions received from the regioncentroid obtaining section 105.

Subsequently, the multiplicity obtaining section 106 adds (step S5) theunprocessed region 1 thus selected to the list of the marker candidateregion list storage section 108. In other words, the marker candidateregion list denotes a list of the regions selected by the multiplicityobtaining section 106, and the candidates of the marker on which thedetermination of whether or not it is the marker is performed by themarker determination section 107.

Subsequently, the region centroid multiplicity obtaining section 106resets (step S6) the multiplicity stored in the multiplicity storagesection in the multiplicity obtaining section 106 to 1.

Subsequently, the region centroid multiplicity obtaining section 106determines whether or not the processing of all of the unprocessedregions has been completed (step S7).

If it is determined in the step S7 that the processing of all of theunprocessed regions has not been completed (No in the step S7), theregion centroid multiplicity obtaining section 106 selects (step S8) oneunprocessed region 2 other than the unprocessed region 1 selected in thestep S4.

Subsequently, the region centroid multiplicity obtaining section 106compares the centroid coordinates of the unprocessed region 1 selectedin the step S4 and the unprocessed region 2 selected in the step S8 witheach other. Further, the region centroid multiplicity obtaining section106 determines (step S9) whether or not the centroid coordinates of theunprocessed region 1 and the unprocessed region 2 overlap with eachother within a predetermined tolerance as a result of the comparison ofthe centroid coordinates. The predetermined tolerance is a value setbased on, for example, the focal length of the camera 120 used forshooting, formation accuracy in forming the marker, the resolution ofthe printer for printing the marker, and so on.

If it is determined in the step S9 that the centroid coordinates of theunprocessed region 1 and the unprocessed region 2 overlap with eachother in the predetermined tolerance (Yes in the step S9), the regioncentroid multiplicity obtaining section 106 adds 1 to the multiplicityof the centroid, and then stores (step S11) it in the multiplicitystorage section in the multiplicity obtaining section 106.

Further, the region centroid multiplicity obtaining section 106 storesthe centroid coordinate and the multiplicity of the unprocessed region 2into the marker candidate region list storage section 108.

Here, as shown in FIG. 10, there are stored the label numbers, thecentroid coordinates, and the multiplicity of the centroid correlatedwith each other.

On the other hand, if it is determined in the step S9 that the centroidcoordinates of the unprocessed region 1 and the unprocessed region 2 donot overlap with each other within the predetermined tolerance (No inthe step S9), the region centroid multiplicity obtaining section 106returns the process to the step S7.

It should be noted that the steps S7 through S11 correspond to acentroid position multiplicity obtaining process.

Subsequently, the marker determination section 107 determines (step S12)whether or not the multiplicity of the centroid of each of the markercandidate regions, namely the unprocessed region 1 and the unprocessedregion 2, stored in the marker candidate region list storage section 108is equal to or larger than a predetermined value, for example, 3. If itis determined in the step S12 that the multiplicity of the centroid ofeach of the candidate regions is equal to or larger than thepredetermined value (Yes in the step S12), the marker determinationsection 107 sets (step S13) the combination of the unprocessed region 1and the unprocessed region 2 to be “processed.”

On the other hand, if it is determined in the step S12 that themultiplicity of the centroid of each of the candidate regions is neitherequal to nor larger than the predetermined value (No in the step S12),the marker determination section 107 returns the process to the step S2.

Subsequently, the marker determination section 107 determines the markercandidate with the multiplicity of the centroid equal to or larger thanthe predetermined value as the marker, and then outputs (step S14) theresult to the marker position output section 109.

It should be noted that the steps S12 through S14 correspond to a markerdetermination process.

The steps S2 through S14 are repeated to obtain the multiplicity of allof the combinations of the labeled regions, and further, whether or notthe combination is the marker is determined.

The marker information determined as the marker by the markerdetermination section 107 is input to the marker position output section109. Further, the marker position output section 109 generates the imageinformation to be displayed on the image display device based on themarker information thus received, and then outputs the image informationthus generated to the image display device 121.

Here, as shown in FIG. 9, the centroid coordinate determined as themarker, the multiplicity of the centroid, the label manes as the regioninformation of the regions having the centroids overlapping with eachother, and so on are output while being correlated to each other.

According to the process described above, the marker processing isterminated.

It should be noted that although the method of performing the detectionof the marker every time two regions are compared with each other ishereinabove described, the method of detecting the marker is not limitedthereto, but it is also possible that the multiplicity obtaining section106 performs the comparison with respect to all of the combinations ofthe regions in advance, and then sequentially obtains the multiplicityusing the result of the comparison. Further, it is also possible toarrange that the marker determination section 107 determines the markerafter all of the multiplicity values have been obtained.

Then, a specific example of the marker process will be explained withreference to FIGS. 11 through 13. FIG. 11 is a diagram showing anexample of an image obtained by binarizing the input image according tothe first embodiment. FIG. 12 is a diagram for explaining the labelingprocess according to the first embodiment. FIG. 13 is a diagram showingan example of a result of obtaining the centroids of the regions havingthe same label according to the first embodiment.

The case in which the image obtained by simplifying the image shot bythe camera 120 in order for explaining the marker process, and thenbinarizing by the binarization section 102 is as shown in FIG. 11 willbe explained.

The region labeling section 104 labels the image information binarizedby the binarization section 102 into constituents 10 through 21 as shownin FIG. 12. It should be noted here that the case in which thebackground 10 is also labeled as a constituent will be explained.

Subsequently, the region centroid obtaining section 105 obtains thecentroid coordinate of each of the labels thus labeled by the regionlabeling section 104. As shown in FIG. 13, the centroids of therespective labels obtained by the region centroid obtaining section 105are denoted as the reference numerals 30 through 41. In FIG. 13, thecoordinate 30 corresponds to the centroid coordinate of the label 10,and the coordinate 31 corresponds to the centroid coordinate of thelabel 11, for example. It should be noted that each of the centroids isindicated by the cross (x) in FIG. 13.

Subsequently, the region centroid multiplicity obtaining section 106sequentially compares the centroid coordinates obtained by the regioncentroid obtaining section 105 to thereby proceed with calculation ofthe multiplicity of the coordinate at which the centroids overlap witheach other. In FIG. 13, the region centroid multiplicity obtainingsection 106 proceeds with sequentially comparing the centroidcoordinates 30 through 41 of the respective labels 10 through 21. InFIGS. 11 and 13, it is determined that the centroid coordinates 31through 33 of the labels 11 through 13 overlap with each other, and themultiplicity is set to 3. Further, it is determined that the centroidcoordinates 38 through 41 of the labels 18 through 21 overlap with eachother, and the multiplicity is set to 4. On the other hand, each of thecentroid coordinates 30, 34 through 37 of the labels 10, 14 through 17does not have a coordinate overlapping therewith, and therefore, themultiplicity is set to 1.

Subsequently, the marker determination section 107 determines whether ornot the multiplicity of the centroid of each of the marker candidateregions stored in the marker candidate region list storage section 108is equal to or larger than a predetermined value, for example, 3. InFIG. 13, the marker determination section 107 determines the regionswith the multiplicity of the centroid equal to or larger than 3, namelythe two regions, one corresponding to the labels 11 through 13 (thecentroids 31 through 33, the multiplicity of the centroid is 3) and theother corresponding to the labels 18 through 21 (the centroids 38through 41, the multiplicity of the centroid is 4), as the markers.Further, the marker determination section 107 outputs the centroidcoordinate (an X coordinate and a Y coordinate of the marker) of theregion determined to be the marker, the multiplicity (the markermultiplicity) of the centroid, and the numbers of the labels as theconstituents of the marker to the marker position output section 109while correlating them with each other.

According to the marker process described above, the shot image isseparated into the markers and the figures other than the markers. Here,in FIG. 12, the labels 11 through 13 and the labels 18 through 21 aremarkers, while the labels 10, 14 through 17 are figures other than themarkers.

Then, the fact that the marker and the marker processing methodaccording to the first embodiment are not influenced by the rotation orthe tilt of the marker, and further by the size (magnification andreduction) of the marker will be explained with reference to FIGS. 14Athrough 14C and 15A through 15C. FIGS. 14A through 14C are diagrams forexplaining the fact that the marker detection does not depend on theposture. FIGS. 15A through 15C are diagrams for explaining the fact thatthe marker detection does not depend on the size.

FIG. 14A is a diagram showing an example in which the markers have notilt, while FIGS. 14B and 14C are diagrams showing examples in which themarkers are tilted (rotated). FIGS. 15A through 15C are diagrams showingan example of the case in which the size of the marker is changed bymagnification or reduction. According to the marker process of the firstembodiment, since the information of a point, namely the centroidcoordinate of the marker, is obtained, and the marker determination isperformed based on the overlap of the centroid coordinates, even in thecase in which the marker is rotated or tilted, or the marker ismagnified or reduced to be changed in size as shown in FIGS. 14A through14C, and 15A through 15C, the marker detection does not depend on theposture (position, rotation, and tilt) of the marker, or the size of themarker.

Then, examples of other markers according to the first embodiment areshown in FIGS. 16A through 16F, 17A through 17F, and 18A through 18D.

FIGS. 16A through 16F are diagrams showing other markers according tothe first embodiment. FIGS. 16A through 16F are diagrams showingexamples of a marker with a fourfold centroid. It should be noted thateach of the centroids is indicated by the cross (x) in FIGS. 16A through16F. As shown in FIGS. 16A through 16F, according to the markerprocessing method of the first embodiment, a variety of markers can beformed.

Since the only requirement of the marker and the marker process in thefirst embodiment is that a predetermined number of centroids of theregions (labels) of the element overlap with each other, if a redundantelement is embedded in a space other than the region, substantially thesame advantage can be obtained. Therefore, examples of the markershaving one or more redundant elements embedded therein are shown inFIGS. 17A through 17F. FIGS. 17A through 17F are diagrams showingexamples of the markers including one or more redundant portions. Asshown in FIGS. 17A through 17F, FIGS. 17A and 17E show examples of afourfold centroid, FIGS. 17B, 17C, and 17F show examples of a threefoldcentroid, and FIG. 17D shows an example of a fivefold centroid. Further,in FIGS. 17A through 17F, reference numerals 51 through 72 denote theregions as the constituents of the markers. For example, in FIG. 17A,the regions as the constituents are denoted by the reference numerals 51through 54, and the centroids of the four regions overlap with eachother at one point (the multiplicity is equal to 4).

Further, FIGS. 18A through 18D are diagrams showing examples of ahandwritten marker. As shown in FIGS. 18A through 18D, according to thefirst embodiment, the marker process can be performed in a similarmanner even in the case of the handwritten markers if the centroids fallwithin a predetermined tolerance, or by setting the tolerance of thecentroid to the range, which the error of the centroids of thehandwritten markers fits in.

As described above, according to the first embodiment, the binarizationsection 102 binarizes the shot image, the region labeling section 104performs the labeling on the image information thus binarized, and thenthe region centroid obtaining section 105 obtains the centroid (thecenter of figure can also be adopted) of each of the regions thuslabeled. Subsequently, the region centroid multiplicity obtainingsection 106 compares the centroids of the regions having the respectivelabels thus obtained to thereby obtain the multiplicity representing howthe centroids overlap with each other. Further, the marker determinationsection 107 performs the marker determination based on whether or notthe multiplicity thus obtained is equal to or larger than apredetermined value. Therefore, it becomes possible to provide a markerindependent of the posture (position, rotation, and tilt) of the markerand further the size of the marker, and to perform the marker process.

Second Embodiment

Then, a second embodiment will be explained with reference to FIGS. 19through 21, 22A through 22L, and 23A through 23L. In the firstembodiment, the marker determination is performed with respect to theregions labeled by the region labeling section 104 using themultiplicity as a degree of overlap of the centroids of the respectiveregions. In the second embodiment, identification of a type of themarker is further performed with respect to the regions labeled by theregion labeling section 104.

FIG. 19 is a block diagram showing an example of a configuration of themarker processing device according to the second embodiment. The markerprocessing device 100 is composed of an image data acquisition section101, a binarization section 102, a binarization threshold settingsection 103, a region labeling section 104, a region centroidcalculation section 105, a region centroid multiplicity calculationsection 106, a marker determination section 107, a marker candidateregion list storage section 108, a marker type identification section201, and a marker position/type output section 202. Further, the imageshot by the camera 120 is input to the marker processing device 100.Further, the marker processing device 100 outputs the marker informationdetected to the image display device 121. The marker processing device100 is different from that of the first embodiment in the marker typeidentification section 201, and the marker position/type output section202.

The marker type identification section 201 receives the information ofeach of the regions obtained in the labeling process from the regionlabeling section 104.

FIG. 21 is a diagram showing an example of the information obtained inthe labeling process by the region labeling section 104. As shown inFIG. 21, due to the labeling process by the region labeling section 104,there are obtained all of the label numbers used for the labelingprocess, the centroid coordinate of each of the labeled regions, thearea of each of the labeled regions, the largest X coordinate of each ofthe labeled regions, the largest Y coordinate of each of the labeledregions, the smallest X coordinate of each of the labeled regions, andthe smallest Y coordinate of each of the labeled regions. It should benoted that the centroid can be the center of figure.

Further, the marker type identification section 201 performsidentification of the marker based on the ratio of the size between theregions, or the ratio of the area between the regions using the regioninformation received from the region labeling section 104. The area ofthe labeled region corresponds to, for example, the total number ofpixels having the same label in the image thus obtained. Further, bysequentially comparing the coordinate values of each of the pixelshaving the same label with the largest value and the smallest value, thelargest value and the smallest value of the coordinate in a certainlabel can be obtained. The area of the region of each label is obtainedby calculation using the largest value and the smallest value of thecoordinate, and the number of pixels having the same label.

The marker determination section 107 firstly performs the markerdetermination based on the multiplicity obtained by the region centroidmultiplicity obtaining section 106.

The marker type identification section 201 identifies the type of themarker with respect to the information with which the regions aredetermined as the marker in the marker determination section 107 usingat least one of the multiplicity of the marker, the area ratio betweenthe regions of the marker, and the ratio of the size of the regions ofthe marker.

Further, the marker type identification section 201 outputs the markerinformation thus determined and identified to the marker position/typeoutput section 202.

The position information of the marker, the type information of themarker, the multiplicity, the label number for constituting the marker,and so on identified by the marker type identification section 201 areinput to the marker position/type output section 202, and the markerposition/type output section 202 generates the information to bedisplayed on the image display device based on the marker informationthus received, and then outputs it to the image display device 121.

FIGS. 22A through 22L are diagrams for explaining an example ofidentifying the marker based on the difference in multiplicity betweenthe markers. In FIGS. 22A through 22L, FIGS. 22A through 22D showexamples of the marker with the multiplicity of 3, FIGS. 22E through 22Hshow examples of the marker with the multiplicity of 4, and FIGS. 22Ithrough 22L show examples of the marker with the multiplicity of 5. Forexample, in the case in which there are three detection objects to beidentified, if the markers with the multiplicities different from eachother are attached respectively, the each of the markers can beidentified, and therefore, the positions of the detection objectsattached with the markers and the types of the detection objectscorrelated with the markers in advance can also be identified. Further,even in the case in which the detection objects and the markers are notcorrelated with each other, it is possible to obtain the position byidentifying the markers with the multiplicities different from eachother based on the shot image.

FIGS. 23A through 23L are diagrams for explaining an example ofidentifying the marker type using the area ratio, the ratio of the sizeof the region of the marker. FIGS. 23A through 23L each show an exampleof the marker with the multiplicity of 3. For example, the three markersshown in FIGS. 23D, 23H, and 23L have the same size, but are differentin size of the region area of each label. Therefore, the markers shownin FIGS. 23D, 23H, and 23L can be distinguished by comparing the areabetween the regions. Similarly, in other markers, it is possible toidentify each of the markers by comparing the ratio of the area betweenthe regions or the ratio of the size between the regions.

Then, the marker processing method according to the second embodimentwill be explained using the flowchart shown in FIG. 20. It should benoted that the explanation of the operations the same as those of thefirst embodiment will be omitted. In FIG. 20, the process of the stepsS201 through S207 is the same as the process of the steps S1 (steps S101through S105) through S14 of the first embodiment.

Then, to the marker type identification section 201, there are input themarker information thus determined from the marker determination section107 and the region labeling information from the region labeling section104. Further, the marker type identification section 201 identifies themarker (a marker type identification process: step S208) based on atleast one of the multiplicity of the marker, the area ratio between theregions of the marker, and the ratio of the size between the regions ofthe marker using the received marker information determined by themarker determination section 107, and the received region labelinginformation labeled by the region labeling section 104.

Subsequently, the marker type identification section 201 outputs (amarker position/type output process: step S209) the marker information(the centroid coordinate of the marker, the label number constitutingthe marker, the multiplicity, and the marker type) thus identified tothe marker position/type output section 202.

According to the process described above, the marker process accordingto the second embodiment is terminated.

As described above, according to the second embodiment, the binarizationsection 102 binarizes the shot image, the region labeling section 104performs the labeling on the image information thus binarized, and thenthe region centroid obtaining section 105 obtains the centroid of eachof the regions thus labeled. Subsequently, the region centroidmultiplicity obtaining section 106 compares the centroids of the regionshaving the respective labels thus obtained to thereby obtain themultiplicity representing how the centroids overlap with each other.Further, the marker determination section 107 performs the markerdetermination based on whether or not the multiplicity thus obtained isequal to or larger than a predetermined value. Further, the marker typeidentification section 201 identifies the marker using the informationgenerated by the region labeling section 104 in the labeling process,the multiplicity obtained by the region centroid multiplicity obtainingsection 106, and the marker information determined by the markerdetermination section 107.

Further, according to the second embodiment, since the centroids of theregions as the marker or the information as a plane is used, it isexpected that the noise component is reduced or averaged by integration,and therefore, the noise resistance higher than that of the marker andthe marker processing method of the related art can be achieved.

As described above, according to the second embodiment, since it isarranged to identify the marker based on the multiplicity of thecentroid (the center of figure can also be adopted) of the marker, thearea ratio between the regions of the marker, or the ratio of the sizebetween the regions, it becomes possible to provide the marker or toperform the marker process independent of the posture (position,rotation, and tilt) of the marker, and of the size of the marker, andfurther having sufficient resistance properties to the noise in the shotimage.

Third Embodiment

Then, a third embodiment will be explained with reference to FIGS. 24through 27. The third embodiment relates to a method of forming themarker embedded with additional information. As explained with referenceto FIGS. 17A through 17F of the first embodiment, since the onlyrequirement of the marker, the marker processing method, and the markerprocessing device of the invention is that a predetermined number ofcentroids of the regions (labels) of the element overlap with eachother, if a redundant element is embedded in a space other than theregion, substantially the same advantage can be obtained.

An example of a method of embedding the redundant information into themarker will be explained with reference to FIGS. 24 through 27. FIG. 24shows a flowchart of a process of embedding additional information intothe marker. FIG. 25 is a diagram for explaining a method of forming aprotective area-added marker. FIG. 26 is a diagram for explaining amethod of adding data to the marker. The process of forming a marker,and of embedding additional information into the marker is performed by,for example, a computer, and is specifically executed along a programstored in a storage device such as a read only memory (ROM) connected toa central processing unit (CPU), a hard disk drive (HDD), or a USBmemory connected via the universal serial bus (USB) I/F.

Firstly, in FIGS. 24 and 25, marker element regions are read out (stepS301) from the marker element region storage section.

Then, the marker element regions used for marker formation are selected(a marker element region selection process: step S302) in the markerelement regions thus read out from the marker element region storagesection. The marker element regions are selected by the computer forperforming the marker formation in a random manner, or based on apredetermined selection criterion, and for example, the marker elementregions denoted by (b) and (d) in FIG. 25 are selected.

Subsequently, as indicated by (e) in FIG. 25, at least one of amagnification process, a reduction process, and a rotation process isperformed (a marker element region transformation process: step S303)alone or in combination with respect to the marker element regions thuselected if necessary. For example, in FIG. 25, the transformationprocess by magnification is performed on the selected element region(b), and the transformation process by reduction, and reduction androtation is performed on the selected marker element region (d).Further, it is also possible to set the marker and the protective areaof the marker so that the size of the marker region becomes the range inwhich the correction of the missing data in the additional informationto be embedded such as a two-dimensional code is possible. Theprotective area of the marker denotes the area in which the image to beembedded is not disposed.

Subsequently, detection of the minimum region width is performed (aminimum region width detection process: step S304) with respect to eachof the marker element regions transformed in the step S303. This is aguard for separating the marker element from the additional informationthus embedded and for protecting the marker element when embedding theadditional information into the marker element region, and the minimumregion width is a region width set based on, for example, the resolutionof the camera 120, the resolution of the printer used when printing themarker, and the size of the marker intended to be formed, and so on.

Subsequently, as indicated by (f) in FIG. 25, the minimum region widthdetected in the step S304 is overlapped (a protective area overlappingprocess: S305) on each of the marker element regions transformed in thestep S303.

Subsequently, as indicated by (g) in FIG. 25, the marker element regionson which the protective area is overlapped in the step S305 are combinedso that the coordinates of the centroids (the center of figure can alsobe adopted) are substantially equal to each other to dispose theprotective area-added marker (a protective area-added marker dispositionprocess: step S306).

Subsequently, in FIGS. 24, 26, the data to be embedded into the markeris obtained (an embedded data obtaining process: step S307). The data tobe embedded can be arranged to be obtained by reading out the datastored in the storage device such as a ROM, an HDD, or a USB memoryconnected via the USB I/F, or obtained via a USB memory or a network.

Subsequently, as shown in the part (a) of FIG. 26, the redundant data isgenerated (a redundant data generation process: step S308) using theembedded data obtained in the step S307. As a redundant data generationmethod, for example, a predetermined method such as a generation methodof a typical two-dimensional code is used.

Subsequently, the redundant data generated in the step S308 issuperimposed into the protective area-added marker disposed in the stepS306 to thereby form (a protective area-added marker superimposingprocess: step S309) the marker embedded with the redundant data.Specifically, the redundant data shown in the part (a) of FIG. 26 issuperimposed into the protective area-added marker shown in the part (b)of FIG. 26 (the same as (g) in FIG. 25), thereby forming the markershown in the part (c) of FIG. 26 embedded with the redundant data.

Subsequently, the protective area is removed from the marker formed inthe step S308 embedded with the redundant data to thereby perform themarker formation, and then, the marker thus formed is output (a markeroutput process: step S310) to the storage device such as a ROM, an HDD,or a USB memory connected via the USB I/F, or a printer connected to thecomputer.

According to the process described above, the formation of the markerembedded with the additional data is terminated.

Then, some examples of embedding additional data into the marker will beexplained with reference to FIGS. 27A through 27E. FIGS. 27A through 27Eare diagrams showing examples of an area of the marker where theadditional data can be embedded, and examples of actually embedding theadditional data. FIG. 27A is a diagram showing an example of the markerwith the multiplicity of 3, and FIG. 27B is a diagram showing an exampleof embedding the additional data into the marker shown in FIG. 27A.Similarly, FIG. 27C is a diagram showing an example of the marker withthe multiplicity of 3, and FIG. 27D is a diagram showing an example ofembedding the additional data into the marker shown in FIG. 27C. FIG.27E shows the marker with the multiplicity of 4, and shows an example ofa range for arbitrary data as the additional data.

As described above, according to the third embodiment, since it isarranged that the protective area is provided to the marker to be formedto thereby superimpose the additional data, it becomes possible to embedthe additional data into the marker within the range where theadditional data can be disposed so as not to affect the marker detectionprocess. As described above, in the case of using the marker processingmethod and the marker according to the invention, a variety of markerscan be formed as described in the first and second embodiments, andmoreover, it is also possible to embed additional information using themethod described in the third embodiment, which provides a wideapplication range and a wide range of practical use.

Further, according to the third embodiment, it is possible to form amarker having a figure representing the additional data disposed so asto surround the constituents of the marker.

It should be noted that although in the third embodiment the method offorming the marker and embedding the additional information into themarker using the computer is explained, the formation of the marker andthe embedding of the additional information into the marker can also beperformed by a marker formation device provided with a function ofperforming the process shown in FIGS. 24 through 26.

Fourth Embodiment

Then, a fourth embodiment will be explained with reference to FIGS. 28Aand 28B. The fourth embodiment relates to an object having the markerformed by the method according to the third embodiment. FIGS. 28A and28B are diagrams showing examples of the object having the markeraccording to the present embodiment. FIG. 28A is a diagram of an exampleof attaching a marker 302 to a screwdriver 301, and FIG. 28B is adiagram of an example of attaching a marker 312 to a spanner 311. Themarkers provided to the objects can have the multiplicities differentfrom each other, and the different ratios of the size between theregions or the different area ratios of the regions as described in thesecond embodiment. Further, by correlating the markers 302, 312 with thescrewdriver 301 and the spanner 311, respectively, and then previouslyregistering them in the marker processing device, the objects can beidentified by performing the identification of the marker.

The identification of the markers is performed using the method of thefirst embodiment, in which the multiplicity of all of the combinationsof the labeled regions is obtained, and further, whether or not thecombination is the marker is determined. Further, according to themethod of the second embodiment, the identification of the markers isperformed based on the ratio of the size between the regions or the arearatio between the regions. Further, even in the case in which aplurality of objects provided with the marker exists in the shot image,and further, the markers are rotated or scaled due to the arrangement ofthe objects as shown in FIG. 5, the identification can be performed withaccuracy. Further, the markers 302, 312 can be bonded to the objects301, 311, or printed on the objects, or formed thereon whenmanufacturing the objects using a metal mold.

As described above, according to the fourth embodiment, since it isarranged that the markers formed by the method of the third embodimentare provided to the objects, and the markers are identified by themethods of the first embodiment and the second embodiment, it ispossible to provide an object having a marker, which is independent ofthe posture and the size of the marker, highly resistant to the noise inthe image, and allowing reduction of an amount of processing for markerdetection by using a monochrome image instead of hue information.

Fifth Embodiment

Then, a fifth embodiment will be explained with reference to FIG. 29.The fifth embodiment relates to another specific example of an objectprovided with a marker. FIG. 29 is a diagram showing an example of theobject having the marker according to the present embodiment. In FIG.29, the object (e.g., a book or a magazine) 321 has the marker 322printed in a page thereof. The marker 322 is formed according to thethird embodiment, and is embedded with additional information. Themarker processing device is previously registered with a uniformresource locator (url) address or the like on the internet in which theitem or the advertisement related to the page attached with the markeris posted. Then the marker processing device obtains the multiplicity ofall of the combinations of the labeled regions, and further, determineswhether or not the combination is the marker using the method of thefirst embodiment. Further, according to the method of the secondembodiment, the identification of the markers is performed based on theratio of the size between the regions or the area ratio between theregions. Further, the marker processing device reads the two-dimensionalcode, and then accesses the url address or the like corresponding to thetwo-dimensional code thus read, and thus displays the item or theadvertisement related to the information added thereto.

It should be noted that although the case of embedding the additionalinformation is explained as an example of the marker attached to theobject 321, in the case in which a plurality of url addresses or thelike on the Internet different from each other for every marker isregistered in the marker processing device while being correlated withthe respective markers, the additional information can be eliminatedfrom the marker attached to the page. Further, the information to becorrelated with the marker or the additional information embedded intothe marker is not limited to the url address, but can be otherinformation, such as image or character information directly correlatedtherewith.

As described above, according to the fifth embodiment, since the markerformed by the method of the third embodiment is attached to the object,the marker is identified by the methods of the first and secondembodiments, the url address or the like is registered to the markerprocessing device in advance while being correlated with the markerattached to the object or the additional information embedded into themarker, and information is read out based on the information thusregistered and correlated with the marker or the additional informationembedded into the marker, and then displayed, it is possible to providean object having a marker, which is independent of the posture and thesize of the marker, highly resistant to the noise in the image, andallowing reduction of an amount of processing for marker detection byusing a monochrome image instead of hue information.

Further, although in the first through fifth embodiments the example ofusing a monochrome pattern as a marker is explained, the shot image canbe a grayscale image within a range in which the contrast equivalent tothe monochrome pattern can be obtained, or a combination of hues.

Further, although in the first through fifth embodiments the example ofusing circles, rectangles, lines, and so on as the marker is explained,this is not a limitation, but polygons, figures surrounded by curves,and so on can also be adopted providing those figures each provided withthe area and the centroid as a region, and the centroid coordinatesthereof fall within a predetermined tolerance.

It should be noted that it is also possible to execute a part of or thewhole functions shown in FIG. 4 of the first embodiment and FIG. 19 ofthe second embodiment in accordance with a program stored in the storagedevice such as a ROM, an HDD, connected to the CPU not shown of themarker processing device, or a USB memory connected via the USB I/F.

What is claimed is:
 1. A marker comprising: at least three figureshaving a common centroid.
 2. The marker according to claim 1, furthercomprising: additional information.
 3. The marker according to claim 2,wherein the additional information is digital data.
 4. An article ofmanufacture comprising: the marker according to claim 1.